Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

openai: add support for setting the base url #267

Merged
merged 1 commit into from
Feb 5, 2024

Conversation

danbev
Copy link
Contributor

@danbev danbev commented Feb 3, 2024

This commit adds support for setting the base url for the openai client. This is useful for testing and for using the client with a different provider.

The motivation for this came from wanting to try out Perplexities API using llm-chain-openai. For this I needed to set the base url to the one provided by Perplexity, but I could find a way to that with the current implementation.

With the change in this commit, an example can be written like this:

use llm_chain::options;
use llm_chain::options::ModelRef;
use llm_chain::{executor, parameters, prompt};

async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let opts = options!(
         Model: ModelRef::from_model_name("pplx-70b-online")
    );
    let exec = executor!(chatgpt, opts.clone())?;
    let query = "What is the capital of Sweden?";
    println!("Query: {query}\n");
    let res = prompt!("", query,).run(&parameters!(), &exec).await?;
    println!("Perplixity AI:\n{res}");
    Ok(())
}

And the following environment variables need to be set:

export OPENAI_API_KEY=<Perplexity-API-Key>
export OPENAI_API_BASE_URL=https://api.perplexity.ai

This commit adds support for setting the base url for the openai
client. This is useful for testing and for using the client with
a different provider.

The motivation for this came from wanting to try out Perplexities API
using `llm-chain-openai`. For this I needed to set the base url to
the one provided by Perplexity, but I could find a way to that with
the current implementation.

With the change in this commit, an example can be written like this:
```rust
use llm_chain::options;
use llm_chain::options::ModelRef;
use llm_chain::{executor, parameters, prompt};

async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let opts = options!(
         Model: ModelRef::from_model_name("pplx-70b-online")
    );
    let exec = executor!(chatgpt, opts.clone())?;
    let query = "What is the capital of Sweden?";
    println!("Query: {query}\n");
    let res = prompt!("", query,).run(&parameters!(), &exec).await?;
    println!("Perplixity AI:\n{res}");
    Ok(())
}
```
And the following environment variables need to be set:
```console
export OPENAI_API_KEY=<Perplexity-API-Key>
export OPENAI_API_BASE_URL=https://api.perplexity.ai
```

Signed-off-by: Daniel Bevenius <[email protected]>
@williamhogman williamhogman merged commit bb57935 into sobelio:main Feb 5, 2024
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants